Methods for polyp detection

ABSTRACT

Disclosed herein are methods for identifying polyps or lesions in a colon. In some variations, computer-implemented methods for polyp detection may be used in conjunction with an endoscope system to analyze the images captured by the endoscopic system, identify any polyps and/or lesions in a visual scene captured by the endoscopic system, and provide an indication to the practitioner that a polyp and/or lesion has been detected.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNumber PCT/US2017/033675, filed on May 19, 2017, which claims priorityto U.S. Provisional Patent Application No. 62/339,019, filed May 19,2016, the disclosures of each of which are hereby incorporated byreference in their entirety.

BACKGROUND

Colonoscopies are medical procedures that utilize a viewing instrumentto examine the interior surface of a colon, which may be used toidentify anatomical abnormalities that may be precursors to colorectalcancer or other intestinal disorders. The American Cancer Societyrecommends that colonoscopies every 10 years for men and women ofaverage colorectal cancer risk, starting at age 50, but earlier and/ormore frequent colonoscopies are recommend for patients at higher risk,including people with a history of prior polyps or inflammatory boweldisease, or a family history of certain genetic colonic diseases. Duringa colonoscopy, a practitioner scans the interior surface of a colonusing an endoscope (i.e., a colonoscope) to visually identify lesions,erosions, polyps, atypical surface textures or coloration, groovesand/or granularities in the mucosal surface of the colon. Typically, thepatient will ingest a colon preparation solution procedure prior to thecolonoscopy to clear out the contents of their colon. This reduces theamount of stool in the colon so that structures and/or textures on thesurface of the colon can be readily scanned, thereby facilitating theidentification of polyps and/or lesions.

Because the interior surface of the colon has many curves and folds, andthe quality of the bowl preparation varies, it may be difficult toidentify polyps and a practitioner may overlook a polyp or lesion.Furthermore, it is in the interest of both the practitioner and thepatient for the colonoscopy to proceed in an expedient manner.Accordingly, improvements to the accuracy of identifying polyps and/orlesions (e.g., reducing the rate of false positive or false negativeresults) and efficiency of colonoscopies are desirable.

BRIEF SUMMARY

Disclosed herein are methods for identifying polyps or lesions in acolon. In some variations, computer-implemented methods for polypdetection may be used in conjunction with an endoscope system to analyzethe images captured by the endoscopic system, identify any polyps and/orlesions in a visual scene captured by the endoscopic system, and providean indication to the practitioner that a polyp and/or lesion has beendetected. Some methods may comprise analyzing a one or more staticimages or video to identify regions with abnormal structure or patterns,determining the likelihood or probability that such region may have apolyp and/or lesion, and prompting the practitioner to visually inspectthat region more closely. Computer-implemented methods of polypdetection may be performed during at least a portion of the colonoscopyprocedure, in real-time (e.g., in about 30 ms or less). In somevariations, an endoscopic system may comprise a plurality of imagingdevices, for example, one or more front-facing imaging devices, one ormore side-facing imaging devices, and/or one or more rear-facing imagingdevices. Any of the polyp detection methods described herein may be usedto analyze the image data from any one or more of the plurality ofimaging devices and to provide a notification to the practitioner when apolyp is identified. In some variations, the notification may includelocation information (optionally, with navigation instructions to thepolyp) and/or anatomical information about the identified polyp(optionally, an image of the colon wall with the boundaries of the polypoutlined).

One example of a method for detecting polyps may comprise acquiring animage from an imaging device located at a distal portion of anendoscope, identifying surface peaks in the image, identifying clustersof surface peaks based on a predetermined threshold separation distance,selecting a surface peak from each identified cluster, defining a pixelregion around each of the selected surface peaks, comparing imagefeatures in each of said defined pixel regions with image features of aplurality of images containing polyps and image features of a pluralityof images that do not contain polyps, and if an image feature in adefined pixel region matches image features of a plurality of imagescontaining polyps, generating a notification that a polyp has beendetected. In some variations, the step of comparing image features maycomprise computing a histogram of oriented gradients (HOG) to extractsurface peaks from the plurality of images containing polyps (HOG-PI),computing a histogram of oriented gradients (HOG) to extract surfacepeaks from the plurality of images that do not contain polyps (HOG-NPI),computing a histogram of oriented gradients (HOG) of the image enclosedby a defined rectangle (HOG-ROI), comparing HOG-ROI with HOG-PI andHOG-NPI, and if the similarity between HOG-ROI to HOG-PI exceeds apreselected threshold, determining that a polyp is detected. In somevariations, the preselected similarity threshold may be at least 50%similarity. In some variations, generating a notification may comprisetransmitting an image of the detected polyp to a display and optionallyproviding an arrow configured to indicate the location of the polyp withrespect to a distal end of the endoscope.

A method for polyp detection may comprise acquiring an image from animaging module located at a distal portion of an endoscope, identifyingsurface peaks in the image, identifying clusters of surface peaks basedon a predetermined threshold separation distance, defining a pixelregion around each of the selected surface peaks, comparing an imagefeature in each of the defined pixel regions with a corresponding imagefeature of a plurality of images containing polyps and a correspondingimage feature of a plurality of images that do not contain polyps, andif the image feature in a defined pixel region matches the correspondingimage feature of a plurality of images containing polyps, generating anotification that a polyp has been detected. Comparing the image featuremay comprise computing a histogram of oriented gradients to extractsurface peaks from the plurality of images containing polyps (HOG-PI),computing a histogram of oriented gradients to extract surface peaksfrom the plurality of images that do not contain polyps (HOG-NPI),computing a histogram of oriented gradients of the image enclosed by adefined rectangle (HOG-ROI), and if the similarity between HOG-ROI toHOG-PI exceeds a preselected similarity threshold, determining that apolyp is detected. The preselected similarity threshold may be at least50% similarity. In some variations, the image feature may comprise acurvature of a high-contrast edge, and/or spatial frequency. Comparingimage features in each of the defined pixel regions may compriseapplying a convolutional neural network (CNN) to the pixel regions, andcalculating a numerical output based on the CNN for each pixel regionthat indicates whether the pixel region contains a polyp. Comparing animage feature may comprise applying a convolutional neural network (CNN)to each pixel region. Applying a CNN may comprise generating a firstfiltered pixel region by filtering the pixel region with a first filterto identify one or more polyp-like features, generating a secondfiltered pixel region by filtering the first filtered pixel region witha second filter to identify one or more non-polyp features, generating anotification that a polyp has been detected if a second filtered pixelregion of the defined pixel regions has been identified to have a higherincidence of polyp-like features than non-polyp features.

The plurality of images containing polyps and the plurality of imagesthat do not contain polyps may be stored on a remote memory or server.Generating a notification may comprise transmitting an image of thedetected polyp to a display and may optionally comprise providing anarrow configured to indicate the location of the polyp with respect to adistal end of the endoscope. In some variations, the imaging module maycomprise a first side-facing imaging device and a second side-facingimaging device, and acquiring an image may comprise acquiring a firstimage from the first side-facing imaging device and a second image fromthe second side-facing imaging device. Image features in each of saiddefined pixel regions may be compared by applying a first CNN to pixelregions of the first image and applying a second CNN to pixel regions ofthe second image. The endoscope may comprise a front-facing imagingdevice, and acquiring an image may comprise acquiring a third image fromthe front-facing imaging device and comparing image features maycomprise applying a third CNN to pixel regions of the third image.

A method for polyp detection may comprise applying a convolutionalneural network (CNN) to an image of the colon. Applying a CNN to animage may comprise selecting a first set of sub-regions of the image byapplying a first convolution stage of the CNN to the image, the firstconvolution stage comprising a first polyp-positive filter thatidentifies sub-regions of the image containing a polyp-like feature,selecting a second set of sub-regions from the first set of sub-regionsby applying a second convolution stage of the CNN to the first set ofsub-regions, where the second convolution stage may comprise a secondpolyp-positive filter that identifies the incidence of a polyp-likefeature in a sub-region and a polyp-negative filter that identifies theincidence of a non-polyp feature in a sub-region, selecting a third setof sub-regions by identifying sub-regions in the second set ofsub-regions where a ratio of the incidence of the polyp-like feature tothe incidence of the non-polyp feature exceeds a pre-determinedthreshold, and generating an output that indicates the presence of apolyp within the image if the number of sub-regions in the third set ofsub-regions meets or exceeds a pre-determined count threshold.Generating an output may comprise generating an output if the ratio ofthe number of sub-regions in the third set to the number of sub-regionsin the second set meets or exceeds a pre-determined ratio threshold. Thepolyp-like feature may comprise a high-contrast edge having a curve witha radius-of-curvature from about 2 mm to about 7 mm, and/or may comprisea pixel having a local maximum intensity that is located within an innercurve of the high-contrast edge. Alternatively or additionally, thepolyp-like feature may comprise surface peaks identified by calculatinga histogram of oriented gradients of a plurality of polyp-positive colonimages (HOG-PI). The non-polyp feature may comprise low-contrast edgeswith a spatial frequency that exceeds a pre-determined spatial frequencythreshold, and/or surface peaks identified by calculating a histogram oforiented gradients of a plurality of polyp-negative colon images(HOG-NPI). The first polyp-positive filter may be the same as ordifferent from, the second polyp-positive filter. The first convolutionstage and/or the second convolution stage may comprise a low-passfilter. The CNN may be a first CNN, and a polyp detection method mayoptionally comprise applying a second CNN to the image of the colon,where the second CNN may comprise a first convolution stage having athird polyp-positive filter and a second convolution stage having afourth polyp-positive filter and a second polyp-negative filter. Thepolyp-like feature may be a first polyp-like feature and the thirdpolyp-positive filter may identify sub-regions of the image containing asecond polyp-like feature different from the first polyp-like feature.The image may be a first image acquired by a first imaging device, andthe CNN may be a first CNN, and the method may optionally compriseapplying a second CNN to a second image of the colon acquired by asecond imaging device. In some variations, the first imaging device maybe a first side-viewing device and the second imaging device may be asecond side-viewing device.

Also disclosed herein is a detachable imaging device comprising animaging module and a clip attached to the imaging module. The imagingmodule may comprise a housing having a front face, a back face, a firstside-facing imaging element and a second side-facing imaging element andthe clip may be configured to be releasably disposed over a distalportion of an endoscope. The clip may comprise a first engagementportion having a front facing edge, a back facing edge, and a bottomedge, and a second engagement portion having a front facing edge, a backfacing edge, and a bottom edge. A space between the first and secondengagement portions may define an endoscope attachment region and theback facing edges of the first and second engagement portions each havean atraumatic protrusion having a rounded contour along the lengths ofthe back facing edges. The bottom edges of the first and secondengagement portions may each have an atraumatic protrusion having arounded contour along the lengths of the bottom edges. Optionally, theatraumatic protrusions of each of the bottom edges may comprise aninward-facing lip that extends into the endoscope attachment region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A depicts one variation of an endoscope system. FIG. 1B depictsanother variation of an endoscope system. FIG. 1C is a schematicrepresentation of one variation of an imaging system and correspondingprocessor, display, and remote server that may support any of theendoscope systems described herein.

FIG. 1D depicts a perspective view of one variation of a detachableimaging module. FIG. 1E depicts a front view of the detachable imagingmodule of FIG. 1D. FIG. 1F depicts a perspective view of one variationof a detachable imaging module. FIG. 1G depicts a front view of thedetachable imaging module of FIG. 1F.

FIG. 2A is a flowchart depiction one variation of a method for polypdetection. FIG. 2B is a flowchart depiction of one variation of a methodfor comparing image features of a region of interest with image featuresof polyp images and non-polyp images.

FIGS. 3A-3D depict an example of an image that has been analyzed andprocessed in accordance with the method of FIG. 2A.

FIG. 4A is a schematic representation of one variation of a displayformat. FIG. 4B is a schematic representation of another variation of adisplay format.

FIG. 5A is a schematic representation of one variation of a displayformat when a polyp has not been detected. FIG. 5B is a schematicrepresentation of the display format of FIG. 5A when a polyp has beendetected.

FIG. 6A is an image acquired by an endoscope depicting an example of avascular pattern on the internal surface of a colon. FIG. 6B is an imageacquired by an endoscope depicting another example of a vascular patternon the internal surface of a colon.

FIG. 7A depicts one variation of a plot that reflects the scan speed ofan endoscope system along various segments of a colon. FIG. 7B is aflowchart depiction of one example of a method for generating the plotof FIG. 7A.

FIG. 8 depicts one variation of a convolutional neural network (CNN)that may be applied to an image for polyp detection.

DETAILED DESCRIPTION

Described herein are methods for polyp detection. The methods may becomputer-implemented methods comprising computer executable instructionsstored in the memory of a controller or processor.

The methods for polyp detection disclosed herein may be used inconjunction with a variety of endoscopes adapted for scanning theinterior surface of a colon (e.g., colonoscopes). For example, methodsfor polyp detection may be used with endoscope systems comprising asingle imaging device that has a forward-facing view (e.g., a field ofview that extends from the distal end of the elongate body of anendoscope), and may also be used with endoscope systems comprising aplurality of imaging devices with various overlapping and/ornon-overlapping views. In some variations, an endoscope or colonoscopesystem may comprise an elongate body having a proximal portion, a distalportion, and side walls extending between the proximal and distalportions, a first imaging device located at a distal portion of theelongate body and having a field-of-view that extends from the distalend of the elongate body (e.g., a forward view, front-facing), and oneor more imaging devices located along the sidewalls of the elongatebody. The one or more imaging devices located on the sidewall of theelongate body may have field-of-views that extend from the side of theelongate body (e.g., side views, rearward views). For example, anendoscope or colonoscope system may comprise a first side-mounted (e.g.,side-facing) imaging device having a first field-of-view that extendsfrom a first sidewall of the elongate body in a first direction and asecond side-mounted (e.g., side-facing) imaging device having a secondfield-of-view that extends from a second sidewall of the elongate bodyin a second direction that is different from the first direction. Somevariations may optionally comprise a side-mounted imaging device thatmay have a field-of-view that extends rearwardly relative to thefield-of-view of a front-facing imaging device, and/or a side-mountedimaging device that may have a field-of-view that extends above or belowthe elongate body. The viewing angle of the one or more side-mountedimaging devices relative to the longitudinal axis of the elongate bodymay be from about 0 degrees (i.e., parallel or coaxial with thelongitudinal axis of the elongate body) to about 179 degrees, forexample, about 90 degrees, about 75 degrees, about 120 degrees, about135 degrees, etc. The field-of-views of the front-facing and the one ormore side-mounted imaging devices may or may not overlap. In somevariations, at least a portion of the field-of-views of the front-facingand the one or more side-mounted imaging devices may overlap.Field-of-views having some degree of overlap may facilitate thecombination or stitching of multiple images from multiple imagingdevices together to simulate a continuous view. In some variations, thecontinuous view may be a panoramic view having a cumulativefield-of-view of at least about 120 degrees, at least about 135 degrees,at least about 150 degrees or more. In some variations, the one or moreside-mounted imaging devices may be integral with the elongate body,while in other variations, the one or more side-mounted imaging devicesmay be releasably attached to the elongate body.

One example of an endoscope (e.g., colonoscope) system comprising anendoscope with a front-facing imaging device and one or more detachableside-facing imaging devices is depicted in FIG. 1A. Endoscope system 100may comprise an endoscope 102 comprising an elongate body 104 and afront-facing imaging device 106 located at the distal end of theelongate body, and a detachable imaging module 110 comprising a firstside-facing imaging device 112 and a second side-facing imaging device(not shown; located on the side opposite to the first side-facingimaging device). The detachable imaging module 110 may comprise a clipor clamp 116 configured to attach to the sidewalls of a distal portionor length of the elongate body 104. The clip or clamp may attach to theelongate body such that it spans a substantial portion of thecircumference of the elongate body, and in some cases, may span theentire circumference of the elongate body or nearly the entirecircumference of the elongate body. For example, the two sides of theclip or clamp may span more than about 50% of the circumference, or morethan about 60% of the circumference, or more than about 70% of thecircumference, or more than about 80% of the circumference, or more thanabout 90% of the circumference, or more than about 95% of thecircumference, etc. Alternatively or additionally, the detachableimaging module may comprise a sleeve (e.g., an elastic or deflectablesleeve) that it encloses the entire circumference of the outer surfaceof the elongate body. The endoscope 102 may optionally comprise a firstlight emitter 108 located on the distal end of the elongate body andconfigured to provide illumination for the field-of-view of thefront-facing imaging device 106. The detachable imaging module 110 mayalso comprise a second light emitter 114 located adjacent to the firstside-facing imaging device 112 and configured to provide illuminationfor the field-of-view of the side-facing imaging device. A third lightemitter (not shown) may be located adjacent to the second side-facingimaging device. In this variation, the axes of the field-of-view of thefirst and second side-facing imaging devices may be tangential to thesurface of the elongate body 104 and/or perpendicular to thelongitudinal axis of the elongate body, while the axis of thefield-of-view of the front-facing imaging device may be approximatelyparallel to the longitudinal axis of the elongate body.

The light-emitters of the detachable imaging module may comprise one ormore light sources, such as light-emitting diodes (LEDs), located withina housing 111 of the imaging module. Alternatively or additionally, thelight-emitters of the detachable imaging module may comprise one or moreoptical fibers connected to a light source located outside of thehousing 111. For example, the light source may be located at a proximalportion of the endoscope system, and the optical output may be channeledthrough the one or more optical fibers to a distal portion of theendoscope system to the imaging module. The ends of the optical fibersmay be located at an opening in the housing to provide illumination forthe field-of-view for the side-facing imaging device. The optical fibers(along with any other control, power and/or data wires) may be enclosedwithin a cable conduit 113 that is located along the outside of theelongate body 104 and connected to the housing 111 of the detachableimaging module. FIG. 1B depicts another variation of an endoscope system120 comprising an endoscope 122 comprising an elongate body 124 and afirst front-facing imaging device 126, similar to that described abovewith respect to FIG. 1A. The endoscope system 120 may also comprise adetachable imaging module 130 comprising a top-viewing imaging device132 that has a field-of-view that has a view axis that is perpendicularto both the surface elongate body 124 and the longitudinal axis of theelongate body. Optionally, the detachable imaging module 130 may alsocomprise a light emitter 134 located adjacent to the top-facing imagingdevice 132 and configured to illuminate the field-of-view of thetop-facing imaging device 132. The detachable imaging module 120 mayfurther comprise a clip or clamp 136 that attaches to the sidewall atthe distal portion of the elongate body 114.

The shape and contours of the housing, along with the shape and contoursof the clip/clamp of any of the detachable imaging modules describedherein may comprise one or more atraumatic features. For example, thehousing and the clip/clamp may have rounded edges and/or tapers to helppromote smooth motion through the colon, without engaging or catchingthe curves and folds of the interior surface of the colon. The frontface (e.g. distal face) and/or the back face (e.g., proximal face) ofthe housing of a detachable imaging module may comprise a roundedtapered contour where the front portion of the housing is narrower thanthe middle portion of the housing. Optionally, the contours and edges ofthe clip/clamp may also have rounded surfaces and/or tapers to helpprevent engaging or catching the colon wall. Some variations may alsohave similar atraumatic contours on the back face of the housing. FIGS.1D-1E depict a detachable imaging module 140 comprising a housing 141, afirst side-facing imaging device 142 and a second side-facing imagingdevice (not shown; located on the side opposite to the first side-facingimaging device), a first light-emitter 144 and a second light-emitter(not shown; located on the side opposite to the first light-emitter).The detachable imaging module 140 may also comprise a clip/clamp orsleeve 146 coupled to the housing 141 to atraumatically secure theimaging module to an endoscope or colonoscope. The clip 146 may comprisea first engagement portion 152 and a second engagement portion 154, andeach of the engagement portions may have curves that approximate thecurvature of the outer surface of an endoscope. The engagement portionsmay be flexible and resilient so that the gap between them may beenlarged to insert an endoscope therebetween and then once the endoscopeis seated between the engagement portions, they may be inwardly biasedto attach over the endoscope. The engagement portions 152, 154 may spanover the majority of the circumference of the endoscope in order tosecure the imaging module thereto. In some variations, the engagementportions may span over at least about 80% (e.g., about 85% or more,about 95% or more) of the total circumference of the outer surface ofthe endoscope, which may help provide a smoother profile with feweredges or protrusions that may unintentionally engage the interiorsurface of the colon. Spanning a larger portion of the circumference mayalso facilitate secure engagement with the elongate body of theendoscope.

The side edges 153 (i.e., the front facing side edges and/or the backfacing side edges) and bottom edges 155 of the engagement portions mayhave rounded or tapered atraumatic contours, as well as enlarged orflattened contours to help distribute any forces over a larger area oftissue. This may help to reduce the incidence of localized regions ofhigh forces that may result in pinching or engagement of any folds orcurves in the colon. For example, the bottom edges 155 of the clip/clamp146 of FIG. 1D and 1E may have enlarged, rounded contours which may helpdistribute any inward clamping forces across a larger surface, and/orallow for an even distribution of lateral forces as the endoscope systemis moved within the lumen of the colon (e.g., across the interiorsurface of the colon). The enlarged, rounded contours 158 may include alip or inward protrusion 157 located on the inner portion of the bottomedges of the engagement portions, which may help engage the elongatebody of an endoscope between the engagement portions 152, 154. FIGS.1F-1G depicts a variation of a detachable imaging module 160 that may besubstantially similar to the other detachable imaging modules describedabove. The imaging module 160 may comprise a clip or clamp 166 havingfirst and second engagement portions 162, 164, each comprising sideedges and bottom edges 175. The bottom edges 175 may also have enlargedrounded contours 168, but unlike the contours 158 depicted in FIG. 1Ewhich have a lip or inward protrusions (i.e., into the space between thetwo engagement portions) and outward protrusions, the enlarged roundedcontours 168 have outward protrusions but no lip or inward protrusions.Alternatively or additionally, bottom edges may be tapered so that theoutermost portion of the engagement portion edge region is thinner thanan inner portion of the edge region. The side edges on the front-facingside and/or the back-facing side may comprise any one or combination ofthe curves, contours, tapers described above. For example, a back facingside edge of a detachable imaging module clip may comprise one or moreof the atraumatic curves, contours and/or tapers described above to helpfacilitate the withdrawal of a colonoscope with the detachable imagingmodule in the colon. In the event that the detachable imaging module isseparated from the colonoscope while within the colon (e.g., during acolonoscopy), the atraumatic curves, contours and/or tapers on theimaging module may help reduce trauma and/or tissue damage to the colonwall as the imaging module is withdrawn from the colon.

The endoscope systems of FIGS. 1A-1B and 1D-1G may further comprise aprocessor or controller in communication with the front-facing imagingdevice and/or the one or more side-mounted imaging devices, and adisplay in communication with the processor, as depicted in FIG. 1C.Optionally, the processor may be connected to a remote server via wiredor wireless connections. Examples of data transferred from the localprocessor or controller to the remote server may include, but are notlimited to, images of the colon, images of polyps or lesions, colonimages that have been classified as containing a polyp (polyp-positiveimages), colon images that have been classified as not containing apolyp (polyp-negative images), and patient-specific data, such asquality of the bowl preparation, date and time of a colonoscopyprocedure, practitioner notes regarding the procedure and the like.Examples of data transferred from the remote server to the processor mayinclude sets of polyp image data collected over one or more populationsof patients, sets of colon images that do not have polyps or lesions,patient-specific data and the like. Additional variations anddescriptions of endoscope systems in which the polyp detection methodsdescribed herein may be applied are described in co-pending U.S. PatentApplication Pub. No. 2014/0343358, filed May 16, 2014. While the imagingdevices depicted and described above are detachable from the elongatebody of an endoscope or colonoscope, it should be understood that theoptical components of the detachable imaging devices may also beintegrated within and/or fixedly attached to the elongate body.

The imaging devices (front-facing and/or side-facing) may acquire stillimages or may acquire a stream of images (e.g., video) that may betransmitted to the processor for analysis, for example, using polypdetection methods. Polyp detection methods may be stored in a memory ofa controller or processor as computer-executable instructions, forexample. In other variations, polyp detection methods may be implementedin computer hardware, for example, in the form of logic gates (e.g., ina FPGA or ASIC). In the variations described herein, the images from theside-mounted imaging devices are analyzed by the processor or controllerusing polyp detection methods, however, it should be understood thatalternatively or additionally, the images from the front-facing imagingdevice may be analyzed using similar polyp detection methods.

One variation of a polyp detection method is depicted in FIG. 2A. Method200 may comprise acquiring 202 an image from an imaging device. Theimage may be acquired in real-time, or may an image acquired in aprevious imaging session and stored in machine-readable memory. Theprocessor may optionally convert 204 the images to grayscale (i.e.,depending on whether the imaging device acquired the image(s) in coloror in black and white). Method 200 may comprise identifying surfacepeaks 206 in the grayscale image. Surface peaks may represent surfaceregions or points that are located at the top surface or the bottomsurface of a fold, as identified by local intensity extremums (e.g.,minimums or maximums, respectively) in the image. For example, a surfacepeak that is located at the bottom of a fold may be further from theimaging device(s) than a surface peak that is located at the top of afold, and the difference in distance may be determined by calculatingthe intensity difference between the identified surface peaks. A surfacepeak that is located at the top of a fold may be brighter (e.g., moreintense) than a surface peak that located at the bottom of the fold, andthe intensity difference may indicate the distance between the twopeaks. Variations of methods that may be used to identify surface peaksmay include various blob detection methods such as MSER (maximallystable extremal regions). A blob detection method may compriseperforming luminance or intensity thresholding of the image (e.g.,sweeping the luminance or intensity threshold from low/black tohigh/white), identifying “extremal regions” by extracting connectedcomponents, finding a threshold where the extremal regions are stable,and storing the extremal regions as a set of surface peaks. For example,a MSER method may comprise generating a sequence of images from a rawimage, where each image is derived from the raw image by applyingvarying intensity thresholds. In some variations, a thresholded imagemay be derived from a raw image by assigning all pixels below (or above)a first threshold to be white (e.g., maximum intensity) and pixels at orabove (or below) the first threshold to be black (e.g., minimumintensity). A second thresholded image may be generated in a similarfashion, but instead using a second threshold that is different from thefirst threshold, and so on (e.g., monotonically increasing or decreasingthe intensity threshold). A MSER method may further comprise identifyingextremal regions within the raw image by selecting one or morethresholded images that have groups of white pixels that stay nearly thesame through a range of thresholds. These extremal regions maycorrespond to surface peaks.

Alternatively or additionally, surface peaks that are located in closeproximity to each other may be used to approximate the curvature of theinterior surface of the colon. For example, the separation between asurface peak at the top of a fold and a surface peak at the bottom of afold may indicate whether the surface curvature is a fold or a polyp.For example, if the separation between surface peaks is relativelylittle (e.g., below a pre-determined separation threshold) and thedistance between the peaks (e.g., as calculated based on intensity) isrelatively high (e.g., above a pre-determined distance threshold), itmay be that the slope of the surface curve is relatively high. A sharpersurface curve, alone or in combination with other polyp-like features,may indicate the presence of a polyp. If the separation between surfacepeaks is relatively high (e.g., above a pre-determined separationthreshold) and the distance between the peaks (e.g., as calculated basedon intensity) is relatively low (e.g., below a pre-determined distancethreshold), it may be that the slope of the surface curve is relativelylow. A low-slope surface curve may indicate that there is a fold orundulation in the surface, but no polyp.

FIGS. 3A-3D depict an example of an image of an interior surface of acolon that contains a polyp, and are annotated to indicate the effect ofthe method steps depicted in FIG. 2A. FIG. 3A depicts an image where theplurality of circles 300 indicate the surface peaks identified as aresult of step 206. Method 200 may further comprise identifying clusters208 of surface peaks and selecting one surface peak from each cluster. Acluster of surface peaks may be defined as any group of surface peaksthat are no more than a selected or predetermined distance apart fromeach other. For example, the selected or predetermined distance valuemay be from about 0.1 mm to about 20 mm, e.g., about 16 mm, about 17 mm,etc. The selected or predetermined distance value may also be defined interms of image pixels, and may be from about 1 pixel to about 200pixels, e.g., about 50 pixels, about 100 pixels, about 150 pixels.Selecting a surface peak from each cluster may comprise calculating thecenter of gravity of the cluster and identifying the surface peak thatis closest to the calculated center of gravity. Alternatively, thesurface peaks in a cluster may be merged by averaging or computing thecenter of gravity of the cluster, where the average or center of gravityis a cumulative surface peak. Alternatively, selecting a surface peakfrom each cluster may comprise comparing the pixel intensities of thesurface peaks and selecting the surface peak with the highest pixelintensity. The thicker-lined circles 302 in FIG. 3B represent theselected surface peaks as a result of step 208. Next, the method 200 maycomprise defining a rectangle 210 around each of the selected surfacepeaks. A rectangle may have a length m and a width n (m×n), where m andn may be from about 5 mm to about 25 mm, e.g., about 16 mm, about 5 mm,about 10 mm, etc., or in terms of image pixels, m and n may be fromabout 10 pixels to about 100 pixels, e.g., about 40 pixels, about 50pixels, about 60 pixels, about 75 pixels, etc. A rectangle may becentered around a selected surface peak, in some instances. Therectangles may also be squares or any other shape. For example, arectangle may be defined around a selected surface peak (x1, y1) bysetting the vertices of the rectangle at a certain distance d_s awayfrom the selected surface peak (e.g., vertex 1: (x1−d_s, y1−d_s), vertex2: (x1+d_s, y1−d_s), vertex 3: (x1−d_s, y1+d_s), vertex 2: (x1+d_s,y1+d_s)). Distance d_s may be, for example 40 pixels to about 100pixels, e.g., about 60 pixels, for an image size of about 400 pixels by400 pixels. If two or more rectangles overlap each other, they may bemerged to form a single larger rectangle. For example, the boundaries ofthe single larger rectangle may be delineated by setting the coordinatesfor the vertices as the minimum and maximum x-coordinate andy-coordinates across both rectangles (i.e., top edge is aligned alongthe maximum y-value, bottom edge is aligned along the minimum y-value,left edge is aligned along the minimum x-value, right edge is alignedalong the maximum x-value).

In some variations, the shape of the region may be characterized by acluster of pixels that meet certain selection characteristics (e.g., RGBvalues and/or brightness values), which may or may not have apre-defined shape. For example, in some variations, rectangles maydelineate the boundaries of pixel regions that may comprise groups ofpixels that have certain characteristics or features that are correlatedwith the presence of a polyp. Examples of image features that may beused to identify whether an image contains a polyp or not may includesurface peak densities (e.g., number of surface peaks per area ofcolon), high-intensity pixel densities, size and shape of high-contrastedges, spatial frequency of low-contrast edges, RGB values and/orchanges of RGB values across a region, etc. Pixel regions that havesurface peak densities that meet or exceed a pre-determined surface peakdensity threshold, and/or disparate RGB values, and/or curvedhigh-contrast edges that have a radius of curvature below apre-determined curvature threshold (e.g., a sharply curved edge with asmaller radius of curvature) may be correlated with a polyp structure.In addition, oval-shaped and/or rounded edges (e.g., relativelyhigh-contrast edges) that may be fully connected or partially connected,and/or a surface peak located in the vicinity of the oval-shaped and/orrounded edges (e.g., within the inner or concave portion of the roundededges) may also be correlated with a polyp structure. In contrast, lowsurface peak densities, similar RGB values across the region (e.g.,homogenous RGB values), and/or curved edges that have a radius ofcurvature above a pre-determined curvature threshold may be correlatedwith non-polyp structures. Low-contrast edges with high spatialfrequencies may be correlated with non-polyp structures or features,such as vascular patterns on the interior surface of the colon. Regionswith RGB values in the blue or purple range may be considered apolyp-like feature while regions with RGB values in the pink or redrange may be considered a non-polyp feature. FIG. 3C depicts rectanglesdefined by step 210. In some variations, closely clustered rectangles(such as the three rectangles in the lower right quadrant of FIG. 3C)may be combined into a single, larger rectangle, as described above.

Method 200 may then comprise comparing features 212 in the enclosedregion of a rectangle to a database of images with polyps and a databaseof images without polyps. This comparison step 212 may be carried outfor each of the regions enclosed by the rectangles from step 210, andmay be executed in parallel or executed sequentially. Methods ofcomparison may include various learning models, for example, anon-probabilistic binary linear classifier, a non-linear classifier(e.g., applying a kernel function) which may comprise regressionanalysis and clustering methods. Some methods may comprise applying oneor more convolutional neural networks (CNNs) to identify images thathave features correlated with the presence of polyps. One variation of amethod 220 that may be used in step 212 of method 200 is depicted inFIG. 2B. Method 220 may comprise computing a histogram of orientedgradients 222 to extract surface peaks from a set of images containingpolyps (HOG-PI), computing a histogram of oriented gradients 224 toextract surface peaks from a set of images that do not contain anypolyps (HOG-NPI), computing a histogram of oriented gradients 226 of aregion of interest (i.e., the region of the image enclosed in arectangle; HOG-ROI), compare HOG-ROI with HOG-PI and HOG-NPI 228, and ifHOG-ROI is most similar to HOG-PI, then a polyp is determined to belocated within the ROI (step 230). In some variations, the HOG ofpolyp-containing images and non-polyp-containing images may be computedonce at the start of a colonoscopy session and not computed or updateduntil the next colonoscopy session. One variation of a method forcomputing the histogram of oriented gradients of an image (or a regionof interest within an image) may comprise dividing the image or regionof interest within an image into overlapping blocks that each has a 2×2array of cells. For example, an image having a size of 64 by 128 pixelsmay be divided into a 16×16 array of blocks, where 50% of each blockoverlaps with its neighboring block. Each block in the array may have2×2 array of cells, which each cell has a size of 8 by 8 pixels. The HOGmethod may further comprise computing centered horizontal and verticalgradients, computing gradient orientation and magnitudes, and quantizingthe gradient orientation into 9 angular bins (from 0 to 180) accordingto the computed gradient orientation. Various learning models, forinstance, SVM, that compare different image feature characteristics orparameters may be used. FIG. 3D depicts the result of step 212, where apolyp is detected in one rectangle and but not the others. If any polypswere identified in one or more rectangles, method 200 comprisesgenerating a notification 214 to inform the practitioner of the possiblepresence of a polyp in an image. A practitioner may optionally confirmthe presence of a polyp, or instead determine that the method 200yielded a false positive. After an imaging session (e.g., a colonoscopysession), the images that have been determined to be polyp-positive maybe added to the set of images with polyps and the images that have beendetermined to be polyp-negative may be added to the set of imageswithout polyps. The sets of polyp-positive and polyp-negative images maybe updated periodically using newly classified images from the same ordifferent clinic. In some variations, images that have been classifiedby a clinician may be collected across multiple offices, clinics and/orany network of service providers and used to update a database ofpolyp-positive images and polyp-negative images. In this way, the HOG-PIand HOG-NPI values or metrics can be constantly updated.

Additionally or alternatively, polyp detection methods may compriseapplying one or more convolutional neural networks (CNNs) to an acquiredimage to determine whether the image contains a polyp. One example of aCNN that may be applied to an image of the colon (either a static imageor a series of images in a video, in real-time or in post-processingafter a colonoscopy session) is depicted in FIG. 8. The CNN 800 maycomprise a first stage of filters or convolutions 802 and a second stageof filters or convolutions 804. The image(s) acquired by an endoscopemay be an input image 806 to which the CNN is applied to determinewhether the image 806 contains a polyp. Applying the first stage offilters 802 to the input image 806 results in a first set of featuremaps 812, subsampling 803 the first set of features maps results in asecond set of feature maps 813, and applying the second stage of filters804 results in a third set of features maps 814. Integrating the featuremaps resulting from these stages of filtering or convolutions maygenerate a metric (e.g., a numerical score) or output 808 thatrepresents the likelihood that the input image 806 contains a polyp. Themetric or output 808 may be compared with a pre-determined orpre-selected threshold to decide whether the input image contains apolyp or not. For example, if the metric or output meets or exceeds athreshold, the input image is classified as containing a polyp (e.g., isclassified as a polyp-positive image). If the metric or output is lessthan the threshold, the input image is classified as not containing apolyp (e.g., is classified as a polyp-negative image). Optionally, ifthe calculated metric is within a specified range of the threshold(e.g., within a calculation error margin), the system may prompt theclinician to direct their attention to the detected features and toconfirm whether or not the feature is a polyp.

In some variations, the first stage of filters or convolutions and/orthe second stage of filters or convolutions may include the methoddepicted in FIG. 2B. Alternatively or additionally, the first stage offilters or convolutions may select for image features that arecorrelated with polyps (i.e., a polyp-positive filter may select for apolyp-like feature set). For example, the first stage of filters 802 mayidentify regions in the input image 806 that have a HOG (i.e., HOG-ROI)that is similar to the HOG of polyp-positive image(s) (i.e., HOG-PI).Other examples of polyp-positive or polyp-like features may includeoval-shaped and/or rounded edges (e.g., relatively high-contrast edges)that may be fully connected or partially connected, and/or a surfacepeak located in the vicinity of the oval-shaped and/or rounded edges, ahigh-contrast edge having a curve with a radius-of-curvature from about2 mm to about 7 mm, and/or surface peaks identified by calculating ahistogram of oriented gradients of a plurality of polyp-positive colonimages (HOG-PI), etc. A first stage of filters or convolutions mayidentify polyp-like image features. Filters or convolutions may alsoeliminate image regions that have more non-polyp features than poly-likefeatures. Examples of non-polyp features (i.e., that may be selected bya polyp-negative filter or convolution) may include diffuse structureswith relatively low contrast, and/or no surface peaks, low-contrastedges with a spatial frequency that exceeds a pre-determined spatialfrequency threshold (e.g., the spatial frequency of a surface of ahealthy colon wall), surface peaks identified by calculating a histogramof oriented gradients of a plurality of polyp-negative colon images(HOG-NPI), etc. These features may be correlated with non-polypstructures, such as the interior surface of the colon, vascular patternsof the colon surface, and/or residual debris from an imperfect bowelpreparation, etc.). Additional image features that may be correlatedwith non-polyp features may include filters or convolutions may comprisea low-pass filter and/or a discrete or fast Fourier transform. Spectralimage data may also be used to help identify a polyp. For example,regions with RGB values in the blue or purple range may be considered apolyp-like feature while regions with RGB values in the pink or redrange may be considered a non-polyp feature. In some variations,applying filters or convolutions to an image may comprise calculating ametric or numerical output that represents the frequency or incidence ofa selected image feature. For example, the numerical output may indicatethe absolute or relative area of the image over which the selectedfeature has been detected, and/or a ratio of the number of pixels (orimage area) of the detected selected feature to the number of pixels (orimage area) of the image where the selected feature was not detected,and/or the number of instances that the selected feature has beendetected in the image, etc. In some variations, a count metric or outputmay comprise the number of image sub-regions that have more polyp-likefeatures than non-polyp features. If the count metric meets or exceeds apre-determined count threshold, the image may be classified as apolyp-positive image. Alternatively or additionally, a ratio metric oroutput may comprise a ratio of the number of image sub-regions that havemore polyp-like features than non-polyp features to the number of imagesub-regions that have fewer polyp-like features than non-polyp features.If the ratio metric meets or exceeds a pre-determined ratio threshold,the image may be classified as a polyp-positive image.

The first set of feature maps 812 may be sampled to select for pixelgroups or image regions that possess the image features selected by thefirst stage of filters or convolutions. The second set of feature maps813 may represent the pixel groups or image regions that have any degreeof similarity with polyp-positive images, even if the degree ofsimilarity is relatively low (e.g., the frequency or number of detectedincidences of polyp-like features is similar to the frequency or numberof detected incidences of non-polyp features). The second set of featuremaps 813 may be a sub-sample of the first set of feature maps 812. Thesecond set of feature maps 813 may then be filtered by a second stage offilters or convolutions 814 that may identify image regions or pixelgroups that have image features that are different from polyp-negativeimages. Alternatively or additionally, the second stage of filters orconvolutions may identify image regions or pixel groups that do notcontain image features or characteristics (e.g., non-polyp features)correlated with polyp-negative images and contain image features orcharacteristics (e.g., polyp-like features) that are correlated withpolyp-positive images. Image regions that do not have featurescorrelated with the presence of a polyp, and/or images that havefeatures that are correlated with non-polyp structures (e.g., colon wallor folds, vascular patterns, bowl residue) may be selected out byfilters or convolutions. Filters or convolutions may also select imageregions for elimination by identifying image regions that have concavecurves/structure, multiple lines or curves distributed across the image,and/or web-like structures that are often associated with blood vesselsor perfusion, and/or any of the previously described non-polyp imagefeatures. The selection of certain features and elimination of otherfeatures may be achieved by applying one or more of the filters orconvolutions described above.

Applying the second stage of filters or convolutions 814 to the secondset of feature maps 814 may result in an integrated set of maps fromwhich the metric or output 808 may be calculated. As an example, thesecond stage of filters or convolutions may generate a polyp surfacepeak similarity metric that represents the similarity of a set offeature maps to the surface peaks of a polyp-positive image. The secondstage of filters or convolutions may generate a non-polyp surface peaksimilarity metric that represents the similarity of a set of featuremaps to the surface peaks of a polyp-negative image. In some variations,the polyp surface peak similarity metric and the non-polyp surface peaksimilarity metric may be compared to calculate the metric or output 808.For example, if the polyp surface peak similarity metric is greater thanthe non-polyp surface peak similarity metric and the difference exceedsa first pre-determined threshold, the output of the CNN may be that theimage contains a polyp. If the polyp surface peak similarity metric isless than the non-polyp surface peak similarity metric, and thedifference exceeds a second pre-determined threshold, the output of theCNN may be that the image does not contain a polyp. If the difference inthese metrics is below any of the pre-determined thresholds, the systemmay generate a notification to the clinician to examine the image moreclosely in order to determine whether a polyp is present or not.

One or more of the steps of method 200 and method 220 may be implementedin computer-executable instructions. The processor or controller maycomprise a central processing unit (CPU), one or more memories incommunication with the central processing unit, and an input-output(I/O) interface that facilitates the communication between the CPU andany peripheral devices, such as the imaging devices of the endoscopesystem, display, remote server, keyboard, mouse, etc. Image dataacquired by any of the imaging devices may be transmitted to the CPUthrough the I/O interface. Optionally, image data may undergopre-processing in a video box prior to being transmitted to the CPU. Insome variations, raw image data may also be transmitted to the display.Analysis of the image data to detect polyps (e.g., steps 204-212 ofmethod 200 and steps 222-230 of method 220) may be carried out by theCPU. Raw image data, intermediate images (such as those depicted inFIGS. 3A-3D), and computer-executable instructions that correspond tothe method steps depicted in FIGS. 2A and 2B may be stored in the one ormemories and accessed as needed by the CPU. Any visual notifications(e.g., error messages, identification of polyps, navigational cues,etc.) may be generated by the CPU and transmitted to the display via theI/O interface. Images that have been acquired and analyzed in the courseof a colonoscopy may optionally be transmitted to a remote server (e.g.,cloud server) that may categorize the images as “polyp images” (e.g.,polyp-positive image) or “non-polyp images” (e.g., polyp-negativeimage). In some variations, data relating to whether an image generateda false positive or false negative may also be transmitted to the remoteserver. As the database of these images increases (i.e., as more andmore practitioners upload colonoscopy images to the server), the polypidentification method may become more sensitive to structures along thecolon surface that may be polyps or lesions.

Images from a previous session that have been classified aspolyp-positive or polyp-negative may be stored in a local and/or remotedatabase. For example, images that have been classified locally (andoptionally visually confirmed by a clinician) as a polyp-positive imageor a polyp-negative image may be transmitted to a remote or cloudserver. Some systems may optionally incorporate these images in one ormore CNNs for polyp detection, which may facilitate and/or expedite theaccurate detection of polyps. For example, images that have beenclassified as polyp-positive may be used to define (or refine)polyp-positive filters or convolutions in a CNN to help identifyfeatures in a newly acquired image (i.e., an input image) that arecorrelated with, and/or indicate the presence of, one or more polyps.Similarly, images that have been classified as polyp-negative may beused to define (or refine) polyp-negative filters or convolutions in aCNN to help identify features in a newly acquired image that arecorrelated with non-polyp tissue (e.g., features that indicate theabsence of polyps or are indicative of colon surface folds, vascularpatterns or bowl debris). For example, the surface peakfeatures/characteristics (e.g., histogram of oriented gradients or HOGs)of polyp-positive images and/or the surface peakfeatures/characteristics (e.g., HOGs) of polyp-negative images may beimplemented in a filter or convolution stage to generate feature maps ofimage regions or pixel groups, as described previously. In somevariations, the filter or convolutions a CNN may be updated using localimage data (e.g., images acquired during colonoscopies at a singlelocation (e.g., a single office or clinic) and/or may be updated usingimage data aggregated over multiple locations (e.g., a network or groupof clinics or offices). Image data that is used to update CNN filters orconvolutions may include full images (which may or may not be classifiedas polyp-positive or polyp-negative images), selected feature mapsand/or extracted features, subsamples of feature maps or filteredimages, and/or images or feature maps that generated by summing aplurality of images or feature maps. Image data may be uploaded to aremote server where it is stored until the next CNN update. For example,image data may be uploaded to a remote server once a day and/or at thecompletion of a colonoscopy session, and updates to a CNN based on newlyuploaded image data may be transmitted to local CNNs once a day and/orupon user-initiated update commands.

Polyp detection methods may include multiple CNNs to identify a varietyof image features or characteristics that are correlated with thepresence of a polyp in an image, and/or identifying features orcharacteristics that are correlated with the absence of a polyp in animage. For example, a polyp detection method may comprise a first CNNthat evaluates whether an image has one or more regions that have HOGprofiles correlated with polyp-positive images and a second CNN thatevaluates the image for one or more regions that have oval-shaped orrounded high-contrast edges (which may or may not be fully connected).Optionally, there may be a third CNN that evaluates the image for one ormore regions that have diffuse, low-contrast edges, and/or web-likestructures correlated with blood vessels. A polyp detection method maycombine the outputs from the first CNN and the second CNN, and determinethat areas where there is a surface peak located on an oval-shaped edgemay have a polyp. Optionally, the method may comprise filtering out(e.g., classifying as polyp-negative) images or image regions that havebeen determined by the third CNN to be polyp-negative. The outputs ofmultiple CNNs may be weighted, for example, using coefficients that areselected at least in part based on the probability, likelihood, orcorrelation between that particular image feature or characteristic andthe presence (or absence) of a polyp. That is, image features that arehighly correlated with the presence of a polyp may be assigned a higherweight or coefficient while image features that are less correlated withthe presence of a polyp may be assigned a lower weight or coefficient.Optionally, additional CNNs may be used to identify surface peakcharacteristics (e.g., number of peaks, distribution or density ofpeaks, movement of peaks across consecutive frames, etc.) that may becorrelated with the presence of a polyp. Other CNNs may optionally beincluded that detect for any number of polyp-like features and/ornon-polyp features.

Optionally, a polyp detection method may comprise a first CNN forprocessing images from the front-facing imaging device, and a second CNNfor processing images from the side-facing imaging device(s). There maybe individual, separate CNNs for each of the side-facing imagingdevices. In some variations, polyp detection methods may use images onlyfrom the side-facing imaging devices while in other variations, polypdetection methods may use images from both the front-facing andside-facing imaging devices. For example, a polyp detection method maycomprise applying a first CNN on images acquired by a first side-facingimaging device and applying a second CNN on images acquired by a secondside-facing device. If a polyp is detected in an image from aside-facing imaging device, the clinician may be prompted to direct thecolonoscope so that the detected polyp is in the field-of-view of thefront-facing imaging device for closer examination and/or confirmation.

In some variations, a polyp detection method may optionally compriseidentifying characteristics of the polyp and its surrounding colonsurface environment and storing data pertaining to those characteristicsin a memory of the processor. This may allow a practitioner to determinewhether a polyp has been encountered previously, or is a newlyidentified polyp. Examples of polyp parameters that may be stored andused to identify a polyp may include size, shape, light reflectionproperties, circumferential location, longitudinal location (e.g., colonsegment where the polyp is located), surface texture, coloration, etc.The location of a polyp may be computed or estimated based on imageanalysis of travel distance relative to an origin (e.g., motiondetection) and/or anatomical structures (e.g., striated muscle patterns,characteristic curves/bends/flexures, rectum folds, vascular patterns),and reference tags selected by the practitioner. Alternatively oradditionally, an accelerometer or position sensor located at or near thedistal end of the elongate body of an endoscope may be used to determinethe real-time location of the imaging device(s) at the time the polyp isdetected. FIGS. 6A and 6B are examples of various vascular patterns inthe mucosal membrane that may be used to identify the location of apolyp and/or may be used as an origin or reference point. The referencepoint may, for example, be stored in memory when the practitionerpresses a button when the endoscope is located at a desired anatomicallocation. Optionally, the selected reference point may be inked.Examples of polyp environment parameters that may be stored and used tofacilitate the identification of a polyp may include light reflectionproperties, coloration and/or textural properties of the surfacesurrounding the polyp, the presence or absence of other polyps orlesions, the presence or absence of certain anatomical structures,and/or vascular patterns.

Position and movement data computed or estimated based on acquiredimages may also be used to compute the speed at which the imagingdevice(s) are moving at particular regions or lengths in the colonduring a scan. In some variations, the position and/or orientation ofthe imaging device(s) in three dimensional space relative to an originor reference point may be estimated using motion detection methods,and/or optionally, with accelerometer and/or position sensor data.Scanning speed and corresponding location/position data may be stored ina memory of the processor, and the processor may generate a plotrepresenting the scan speed at various colon segments that is displayedto the practitioner. One example of such a plot is depicted in FIG. 7A.In some variations, scanning speeds may be represented by differentcolors at each location of the colon. For example, a green dot (or anydesired marker or indicator) at a particular colon segment may indicatethat the scanning speed was greater than a predetermined high-speedthreshold, and a blue dot may indicate that the scanning speed was lessthan a predetermined low-speed threshold (e.g., completely stopped). Insome variations, a purple colored dot may indicate segments of the colonthat have been scanned more than once. Optionally a third color may beused to represent when the scan speed is within a predetermined range.One variation of a method 700 for generating the position-scan speedplot of FIG. 7A is represented in the flowchart depicted in FIG. 7B. Asdepicted there, method 700 may comprise defining 702 a scan startreference point and a scan end reference point. In some examples, theend reference point may be the same as the start reference point, whilein other examples, the end reference point may be different from thestart reference point. These reference points may be automaticallyassigned by the processor, which may be configured to detect when theendoscope system is inserted into the colon, and/or may be assigned asdesired by the practitioner. Method 700 may further comprise plotting704 the position of the imaging device(s) of the endoscope systemrelated to the starting reference point during the scan. The real-timeposition of the imaging device(s) relative to the starting referencepoint may be determined using any of the methods described above, and/oroptionally using data from a gyroscope and/or accelerometer located onthe elongate body of the endoscope system. The position of the imagingdevice(s) may be represented, for example, by a dot. For example, if theimaging device(s) move to the left/right of the starting referencepoint, a dot is plotted to the left/right of the reference point on theplot, and if the imaging device(s) move forward/backward to thereference point, a dot is plotted forward/backward of the referencepoint, and so on. The method 700 may further comprise computing 706 thescan speed at the positions represented by the dots, and integratingthis data with the position data. For example, the dots may becolor-coded as indicated above, or the transparency or intensity of thedot color may be proportional to the scan speed, etc. The position-scanspeed plot may then be displayed 708 to the practitioner. Optionally,the curvature of the colon and/or the travel trajectory of the endoscopesystem may be approximated by a line on the position-scan speed plot.This plot may provide feedback to the practitioner as to how long thepractitioner spent examining certain segments of the colon, and mayfacilitate the detection of colon segments that the practitioner mayhave missed. The data from a position-scanning speed plot may also beused to facilitate practitioner training, for example, allowing thepractitioner to determine whether they consistently miss inspectingcertain areas of the colon, and/or whether they spend too much timeinspecting other areas of the colon, and may also assist thepractitioner in pacing the scan speed at a desired rate. Theposition-scanning speed plot (such as that depicted in FIG. 7A) may begenerated by the processor and displayed during the scanning session(thereby providing real-time feedback to the practitioner) and/or may begenerated by the processor after the conclusion of the scan. Scanningspeed data (e.g., a position-scanning speed plot) may also betransmitted to a remote server.

When a polyp is detected by the processor or controller, a notificationmay be provided to the practitioner conducting the scan. Various typesof notifications may be used to inform the practitioner of the presenceand location of a polyp. FIG. 4A depicts one variation of a display 400where a first image 402 from a front-facing imaging device is located inthe center of the display with the images from the side-mounted imagingdevices (e.g. first side-facing image 404 and second side-facing image406) located around the first image 402 (i.e., on either side of thefirst image). The images 402, 404, and 406 may be the scaled such thatthey are substantially the same size. FIG. 4B depicts a variation wherethe image 414 from the front-facing imaging device is larger (e.g.,having a greater area, and/or greater length and/or greater width) thanthe images 414, 416 from the side-mounted imaging devices. For thedisplay formats depicted in FIGS. 4A and 4B, the processor may outlinethe edges of detected polyp in whichever image that contains the polyp.For example, if a polyp is detected in right image 406, the edge of thepolyp may be highlighted or outlined in image 406. If the practitionersteers the endoscope so that the polyp is in view of the front-facingimaging device and the polyp appears on the center image 402, then theedge of the polyp may be highlighted or outlined in image 402. In somevariations, movement of the endoscope with respect to the polyp may berepresented by transient shadows to help cue the practitioner as to thedirection of movement and/or orientation of the distal end of theendoscope. FIGS. 5A and 5B depict another variation of a display thatmay be included with any of the endoscope systems described herein. Inthis example, the images from the one or more side-mounted imagingdevices are not included on the display unless a polyp is detected inone of those images. This may help to limit visual clutter for thepractitioner, which may in turn help clarify the orientation of thedistal end of the colonoscope with respect to the colon. While thepractitioner is moving through the colon and no polyp is detected, thepractitioner may have the view as depicted in FIG. 5A. The display 500shows the image 502 as acquired from the front-facing imaging device.Although the display 500 does not depict the images acquired from theone or more side-mounted imaging devices, those devices may becontinuously acquiring image data and the processor may be continuouslyanalyzing image data from the one or more side-mounted imaging devicesto detect polyps. If a polyp is detected in one of the side-mountedimaging devices, the image 504 containing the polyp 504 may appear onthe display 500, and an arrow 506 may appear in the image 502 to helpthe practitioner navigate towards the detected polyp, as depicted inFIG. 5B. The practitioner may have the option of steering the endoscopetoward the detected polyp to acquire further images to confirm that itis a true polyp or lesion, and/or to biopsy or excise the polyp.

Optionally, a processor may provide navigational guidance to apractitioner to provide advanced notice of approaching curves in thecolon. This may help to reduce the likelihood of a practitioneradvancing the distal end of the endoscope into the wall of the colon(which often causes discomfort or pain to the patient). In onevariation, the processor may be configured to identify features in animage that indicate a change in the curvature of the colon lumen, andwhen a change in curvature is detected, an arrow may appear on thedisplay that indicates the direction of the curvature change. In somevariations, the processor may track the movement of the darkest regionof the image, where the darkest region of the image may represent theregion of the colon furthest from the endoscope. If the upcoming lengthof colon is relatively straight, the location of the darkest region ofthe image may remain in a central area of the image as the endoscope isadvanced forward. If the upcoming length of colon curves, the locationof the darkest region of the image may shift away from the central areaof the image as the endoscope is advanced. For example, if the colonsegment curves to the right, the darkest region in the image may movetowards the right. If the area of the image occupied by the dark regionmonotonically grows, the processor may interpret such visual cue as theendoscope is moving in a trajectory that will impact or collide with thecolon wall. The processor may generate a notification to prompt thepractitioner to quickly steer the endoscope in the direction of thecurve. The arrow may flash at a frequency that indicates the proximityof the distal end of the endoscope to the colon wall ahead of it. Forexample, as the distal tip nears the wall, the arrow flashing frequencymay increase. Alternatively or additionally, an audible signal may begenerated if the processor determines that the distal tip of theendoscope is within a predetermined distance from a colon wall. Forexample, the audible signal may be a tone pulsed at an initial frequencyand as the distal tip nears a colon wall and is at risk of directlycontacting the wall, the frequency may increase. A method for providingnavigational guidance may comprise identifying the dark region of animage (e.g., lumen of the colon) from a front-facing imaging device of acolonoscope, determining whether the dark region remains in a centralarea of the image (or field-of-view of the front-facing imaging device)as the colonoscope is advanced, and if the dark region shifts from thecentral area of the image, providing an indication to the clinician tosteer the colonoscope in the direction of the shift. The method mayoptionally comprise determining whether the area occupied by the darkregion monotonically grows as the colonoscope is advanced and providingan indication to the clinician to steer the colonoscope away from thewall of the colon (e.g., steer left or right).

It should be understood that while the polyp detection methods describedabove are employed in the context of an endoscope or colonoscope system,these methods may also be used to analyze images collected by anyimaging system suitable for scanning the internal surface of the colon.For example, polyp detection methods may be used to analyze imagesacquired using capsule or pill-based imaging systems, which may beingested or otherwise inserted into the gastrointestinal tract. Examplesof such systems are described in U.S. Pat. No. 7,039,453.

An endoscope system may comprise a controller in communication with theendoscope and the imaging devices mounted thereon and/or attachedthereto. The controller may comprise one or more processors and one ormore machine-readable memories in communication with the one or moreprocessors. The controller may be connected to the imaging devices bywired or wireless communication channels.

The controller may be implemented consistent with numerous generalpurpose or special purpose computing systems or configurations. Variousexemplary computing systems, environments, and/or configurations thatmay be suitable for use with the systems and devices disclosed hereinmay include, but are not limited to software or other components withinor embodied on personal computing devices, network appliances, serversor server computing devices such as routing/connectivity components,portable (e.g., hand-held) or laptop devices, multiprocessor systems,microprocessor-based systems, and distributed computing networks.

Examples of portable computing devices include smartphones, personaldigital assistants (PDAs), cell phones, tablet PCs, phablets (personalcomputing devices that are larger than a smartphone, but smaller than atablet), wearable computers taking the form of smartwatches, portablemusic devices, and the like, and portable or wearable augmented realitydevices that interface with an operator's environment through sensorsand may use head-mounted displays for visualization and user input.

In some embodiments, a processor may be any suitable processing deviceconfigured to run and/or execute a set of instructions or code and mayinclude one or more data processors, image processors, graphicsprocessing units, digital signal processors, and/or central processingunits. The processor may be, for example, a general purpose processor,Field Programmable Gate Array (FPGA), an Application Specific IntegratedCircuit (ASIC), or the like. The processor may be configured to runand/or execute application processes and/or other modules, processesand/or functions associated with the system and/or a network associatedtherewith. The underlying device technologies may be provided in avariety of component types, e.g., metal-oxide semiconductor field-effecttransistor (MOSFET) technologies like complementary metal-oxidesemiconductor (CMOS), bipolar technologies like emitter-coupled logic(ECL), polymer technologies (e.g., silicon-conjugated polymer andmetal-conjugated polymer-metal structures), mixed analog and digital, orthe like.

In some embodiments, memory may include a database and may be, forexample, a random access memory (RAM), a memory buffer, a hard drive, anerasable programmable read-only memory (EPROM), an electrically erasableread-only memory (EEPROM), a read-only memory (ROM), Flash memory, etc.The memory may store instructions to cause the processor to executemodules, processes and/or functions associated with the system, such asone or more of the polyp detection methods described herein, images tobe analyzed, and previously analyzed and/or classified image data.Alternatively or additionally, the memory may store data relating to oneor more CNNs.

Some embodiments described herein relate to a computer storage productwith a non-transitory computer-readable medium (also may be referred toas a non-transitory processor-readable medium) having instructions orcomputer code thereon for performing various computer-implementedoperations. The computer-readable medium (or processor-readable medium)is non-transitory in the sense that it does not include transitorypropagating signals per se (e.g., a propagating electromagnetic wavecarrying information on a transmission medium such as space or a cable).The media and computer code (also may be referred to as code oralgorithm) may be those designed and constructed for the specificpurpose or purposes. Examples of non-transitory computer-readable mediainclude, but are not limited to, magnetic storage media such as harddisks, floppy disks, and magnetic tape; optical storage media such asCompact Disc/Digital Video Discs (CD/DVDs); Compact Disc-Read OnlyMemories (CD-ROMs), and holographic devices; magneto-optical storagemedia such as optical disks; solid state storage devices such as a solidstate drive (SSD) and a solid state hybrid drive (SSHD); carrier wavesignal processing modules; and hardware devices that are speciallyconfigured to store and execute program code, such asApplication-Specific Integrated Circuits (ASICs), Programmable LogicDevices (PLDs), Read-Only Memory (ROM), and Random-Access Memory (RAM)devices. Other embodiments described herein relate to a computer programproduct, which may include, for example, the instructions and/orcomputer code disclosed herein.

A user interface may serve as a communication interface between anoperator and the endoscope system. The user interface may comprise aninput device and output device (e.g., touch screen and display) and beconfigured to receive input data and output data from one or more of theimaging devices, an input device, output device, network, database, andserver. For example, images acquired by an imaging device may bereceived by the user interface, processed by processor and memory, anddisplayed by the output device (e.g., monitor display). Sensor data fromone or more sensors (e.g., accelerometer, temperature sensor, positionsensor, gyroscope, etc.) may be received by the user interface andoutput visually, audibly, and/or through haptic feedback by one or moreoutput devices. As another example, operator control of an input device(e.g., joystick, keyboard, touch screen) may be received by the userinterface and then processed by the processor and the memory forcontrolling the movement of the endoscope and/or operation of the one ormore imaging devices.

In variations of an input device comprising at least one switch, aswitch may comprise, for example, at least one of a button (e.g., hardkey, soft key), touch surface, keyboard, analog stick (e.g., joystick),directional pad, mouse, trackball, jog dial, step switch, rocker switch,pointer device (e.g., stylus), motion sensor, image sensor, andmicrophone. A motion sensor may receive operator movement data from anoptical sensor and classify an operator gesture as a control signal. Amicrophone may receive audio and recognize an operator voice as acontrol signal. In variations of a system comprising a plurality ofinput devices, different input devices may generate different types ofsignals.

In variations of the input device comprising one or more buttons, buttonpresses of varying duration may execute different functions. Forexample, a lumen output level of a light source may be configured toincrease with a longer button press. Conversely, a shorter durationbutton press may correspond to a different function such as deactivatingthe light source.

In some variations, a system may comprise a plurality of input devicesprovided in separate housings, where for example a first input devicemay be handheld and/or portable while a second input device may bestationary. In some variations, a first input device may comprise atablet including a touch screen display and a second input device maycomprise a step switch or foot pedal. The step switch may in somevariations be a confirmation switch that must be engaged at the sametime as contact with the touch screen before a control signal istransmitted to the surgical system. Output of a control signal uponsimultaneous engagement of a first input device and second input devicemay confirm that operator input to the first input device isintentional.

An output device of an endoscope system may output sensor datacorresponding to a patient and/or endoscope system, and may comprise oneor more of a display device, audio device, and haptic device. The outputdevice may be coupled to a patient platform and/or disposed on a medicalcart adjacent to the patient and/or operator. In other variations, theoutput device may be mounted to any suitable object, such as furniture(e.g., a bed rail), a wall, a ceiling, and may be self-standing.

A display device may allow an operator to view images acquired by theone or more imaging devices. In some variations, an output device maycomprise a display device including at least one of a light emittingdiode (LED), liquid crystal display (LCD), electroluminescent display(ELD), plasma display panel (PDP), thin film transistor (TFT), organiclight emitting diodes (OLED), electronic paper/e-ink display, laserdisplay, and/or holographic display.

An audio device may audibly output patient data, sensor data, systemdata, alarms and/or warnings. For example, the audio device may outputan audible warning when the distal end of the endoscope is detected asapproaching a wall of the colon. As another example, audio may be outputwhen operator input is overridden by the system to prevent potentialharm to the patient and/or endoscope system. In some variations, anaudio device may comprise at least one of a speaker, piezoelectric audiodevice, magnetostrictive speaker, and/or digital speaker. In somevariations, an operator may communicate to other users using the audiodevice and a communication channel. For example, the operator may forman audio communication channel (e.g., VoIP call) with a remote operatorand/or observer.

A haptic device may be incorporated into one or more of the input andoutput devices to provide additional sensory output (e.g., forcefeedback) to the operator. For example, a haptic device may generate atactile response (e.g., vibration) to confirm operator input to an inputdevice (e.g., touch surface). As another example, haptic feedback maynotify that an operator input is overridden by the surgical system toprevent potential harm to the patient and/or system.

In some embodiments, the systems, apparatuses, and methods may be incommunication with other computing devices via, for example, one or morenetworks, each of which may be any type of network (e.g., wired network,wireless network). A wireless network may refer to any type of digitalnetwork that is not connected by cables of any kind. Examples ofwireless communication in a wireless network include, but are notlimited to cellular, radio, satellite, and microwave communication.However, a wireless network may connect to a wired network in order tointerface with the Internet, other carrier voice and data networks,business networks, and personal networks. A wired network is typicallycarried over copper twisted pair, coaxial cable and/or fiber opticcables. There are many different types of wired networks including widearea networks (WAN), metropolitan area networks (MAN), local areanetworks (LAN), Internet area networks (IAN), campus area networks(CAN), global area networks (GAN), like the Internet, and virtualprivate networks (VPN). Hereinafter, network refers to any combinationof wireless, wired, public and private data networks that are typicallyinterconnected through the Internet, to provide a unified networking andinformation access system.

Cellular communication may encompass technologies such as GSM, PCS, CDMAor GPRS, W-CDMA, EDGE or CDMA2000, LTE, WiMAX, and 5G networkingstandards. Some wireless network deployments combine networks frommultiple cellular networks or use a mix of cellular, Wi-Fi, andsatellite communication. In some embodiments, the systems, apparatuses,and methods described herein may include a radiofrequency receiver,transmitter, and/or optical (e.g., infrared) receiver and transmitter tocommunicate with one or more devices and/or networks.

Although the foregoing variations have, for the purposes of clarity andunderstanding, been described in some detail by of illustration andexample, it will be apparent that certain changes and modifications maybe practiced, and are intended to fall within the scope of the appendedclaims. Additionally, it should be understood that the components andcharacteristics of the systems and devices described herein may be usedin any combination. The description of certain elements orcharacteristics with respect to a specific figure are not intended to belimiting or nor should they be interpreted to suggest that the elementcannot be used in combination with any of the other described elements.For all of the variations described above, the steps of the methods maynot be performed sequentially. Some steps are optional such that everystep of the methods may not be performed.

1. A method for polyp detection, the method comprising: acquiring animage from an imaging module located at a distal portion of anendoscope; identifying surface peaks in the image; identifying clustersof surface peaks based on a predetermined threshold separation distance;selecting a surface peak from each identified cluster; defining a pixelregion around each of the selected surface peaks; comparing an imagefeature in each of said defined pixel regions with a corresponding imagefeature of a plurality of images containing polyps and a correspondingimage feature of a plurality of images that do not contain polyps; andif the image feature in a defined pixel region matches the correspondingimage feature of a plurality of images containing polyps, generating anotification that a polyp has been detected.
 2. The method of claim 1,wherein comparing the image feature comprises: computing a histogram oforiented gradients to extract surface peaks from the plurality of imagescontaining polyps (HOG-PI); computing a histogram of oriented gradientsto extract surface peaks from the plurality of images that do notcontain polyps (HOG-NPI); computing a histogram of oriented gradients ofthe image enclosed by a defined rectangle (HOG-ROI); comparing HOG-ROIwith HOG-PI and HOG-NPI; and if the similarity between HOG-ROI to HOG-PIexceeds a preselected similarity threshold, determining that a polyp isdetected.
 3. The method of claim 2, wherein the preselected similaritythreshold is at least 50% similarity.
 4. The method of claim 1, whereinthe plurality of images containing polyps and the plurality of imagesthat do not contain polyps are stored on a remote memory or server. 5.The method of claim 1, wherein the image feature comprises a curvatureof a high-contrast edge.
 6. The method of claim 1, wherein the imagefeature comprises spatial frequency.
 7. The method of claim 1, whereincomparing image features in each of said defined pixel regionscomprises: applying a convolutional neural network (CNN) to said pixelregions; and calculating a numerical output based on the CNN for eachpixel region that indicates whether the pixel region contains a polyp.8. The method of claim 1, wherein comparing an image feature comprisesapplying a convolutional neural network (CNN) to each pixel region,wherein applying the CNN comprises: generating a first filtered pixelregion by filtering the pixel region with a first filter to identify oneor more polyp-like features, generating a second filtered pixel regionby filtering the first filtered pixel region with a second filter toidentify one or more non-polyp features, and wherein generating thenotification that a polyp has been detected comprises generating thenotification if a second filtered pixel region of said defined pixelregions has been identified that has a higher incidence of polyp-likefeatures than non-polyp features.
 9. The method of any one of the aboveclaims, wherein generating a notification comprises transmitting animage of the detected polyp to a display.
 10. The method of claim 9,wherein generating a notification further comprises providing an arrowconfigured to indicate the location of the polyp with respect to adistal end of the endoscope.
 11. The method of claim 1, wherein theimaging module comprises a first side-facing imaging device and a secondside-facing imaging device, and wherein acquiring an image comprisesacquiring a first image from the first side-facing imaging device and asecond image from the second side-facing imaging device, and whereincomparing image features in each of said defined pixel regions comprisesapplying a first CNN to pixel regions of the first image and applying asecond CNN to pixel regions of the second image.
 12. The method of claim11, wherein the endoscope comprises a front-facing imaging device, andwherein acquiring an image comprises acquiring a third image from thefront-facing imaging device and wherein comparing image featurescomprises applying a third CNN to pixel regions of the third image. 13.A method for polyp detection comprising: applying a convolutional neuralnetwork (CNN) to an image of the colon, wherein applying the CNNcomprises: selecting a first set of sub-regions of the image by applyinga first convolution stage of the CNN to the image, the first convolutionstage comprising a first polyp-positive filter that identifiessub-regions of the image containing a polyp-like feature; selecting asecond set of sub-regions from the first set of sub-regions by applyinga second convolution stage of the CNN to the first set of sub-regions,the second convolution stage comprising a second polyp-positive filterthat identifies the incidence of a polyp-like feature in a sub-regionand a polyp-negative filter that identifies the incidence of a non-polypfeature in a sub-region; selecting a third set of sub-regions byidentifying sub-regions in the second set of sub-regions where a ratioof the incidence of the polyp-like feature to the incidence of thenon-polyp feature exceeds a pre-determined threshold; and generating anoutput that indicates the presence of a polyp within the image if thenumber of sub-regions in the third set of sub-regions meets or exceeds apre-determined count threshold.
 14. The method of claim 13, whereingenerating an output comprises generating an output if the ratio of thenumber of sub-regions in the third set to the number of sub-regions inthe second set meets or exceeds a pre-determined ratio threshold. 15.The method of any one of claim 13 or 14, wherein the polyp-like featurecomprises a high-contrast edge having a curve with a radius-of-curvaturefrom about 2 mm to about 7 mm.
 16. The method of claim 15 wherein thepolyp-like feature further comprises a pixel having a local maximumintensity that is located within an inner curve of the high-contrastedge.
 17. The method of any one of claim 13 or 14, wherein thepolyp-like feature comprises surface peaks identified by calculating ahistogram of oriented gradients of a plurality of polyp-positive colonimages (HOG-PI).
 18. The method of any one of claim 13 or 14, whereinthe non-polyp feature comprises low-contrast edges with a spatialfrequency that exceeds a pre-determined spatial frequency threshold. 19.The method of any one of claim 13 or 14, wherein the non-polyp featurecomprises surface peaks identified by calculating a histogram oforiented gradients of a plurality of polyp-negative colon images(HOG-NPI).
 20. The method of any one of claim 13 or 14, wherein thefirst polyp-positive filter is the same as the second polyp-positivefilter.
 21. The method of any one of claim 13 or 14, wherein the firstpolyp-positive filter is the different from the second polyp-positivefilter.
 22. The method of any one of claim 13 or 14, wherein the firstconvolution stage comprises a low-pass filter.
 23. The method of any oneof claim 13 or 14, wherein the second convolution stage comprises alow-pass filter.
 24. The method of claim 13, wherein the CNN is a firstCNN, and the method further comprises applying a second CNN to the imageof the colon, wherein the second CNN may comprise a first convolutionstage having a third polyp-positive filter and a second convolutionstage having a fourth polyp-positive filter and a second polyp-negativefilter.
 25. The method of claim 24, wherein the polyp-like feature is afirst polyp-like feature and the third polyp-positive filter identifiessub-regions of the image containing a second polyp-like featuredifferent from the first polyp-like feature.
 26. The method of claim 13,wherein the image is a first image acquired by a first imaging device,and the CNN is a first CNN, and the method further comprises applying asecond CNN to a second image of the colon acquired by a second imagingdevice.
 27. The method of claim 26, wherein the first imaging device isa first side-viewing device and the second imaging device is a secondside-viewing device.
 28. A detachable imaging device comprising: animaging module comprising a housing having a front face, a back face, afirst side-facing imaging element and a second side-facing imagingelement; and a clip attached to the imaging module, the clip configuredto be releasably disposed over a distal portion of an endoscope, whereinthe clip comprises: a first engagement portion having a front facingedge, a back facing edge, and a bottom edge; a second engagement portionhaving a front facing edge, a back facing edge, and a bottom edge,wherein a space between the first and second engagement portions definean endoscope attachment region; and wherein the back facing edges of thefirst and second engagement portions each have an atraumatic protrusionhaving a rounded contour along the lengths of the back facing edges. 29.The device of claim 28, wherein the bottom edges of the first and secondengagement portions each have an atraumatic protrusion having a roundedcontour along the lengths of the bottom edges.
 30. The device of claim29, wherein the atraumatic protrusions of each of the bottom edgescomprise an inward-facing lip that extends into the endoscope attachmentregion.